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(57) Zusammenfassung: Verfahren zur Bildregistrierung durch iterative Bestimmung einer hinsichtlich eines vorgegebenen Dis- 
tanz- und Glattheitskriteriums optimalen Transformation, bei dem in den Bildern korrespondierende Kontrollpunkte garantierbar 
aufeinander abgebildet werden, durch (1) Initialisieren eines Iterationszahlers und des initialen Verriickungsfeldes, (2) Bestimmen 
der numerischen Uisungen der nicht-linearen partiellen Differentialgleichung (PDE) mit dem aus einem vorgegebenen Glattheits- 
kriterium ableitbarem Differentialope rator und den an vorgegebenen Kontrollpunkten lokalisierten Punktauswertefunktionalen, (3) 
Zusammenfassen der Interpolationsbedingungen, (4) Berechnen einer speziellen numerischen Losung der PDE mit der auf der Ba- 
sis des Distanzkriteriums und des aktuellen Verriickungsfeldes bestimmten Kraft und dem aus dem Glattheitskriterium abgeleiteten 
Differentialoperator, (5) Auswerten der speziellen Losung an den Kontrollpunkten, (6) Bestimmen der Koeffizienten zur Berech- 
nung einer aktualisierten Verriickung, (7) Aufdatieren des Verruckungsfelds und Erhohen des Iterationszahlers, (8) Uberpriifen der 
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1. Field of the Invention 

The invention relates to an image registration or 
recording method, i.e. for the correction of geometrical 
^differences in different representations of an object. These 
methods play an important part, e.g. in medical technology and, 
particularly, when analyzing tissue changes in conjunction with 
early cancer diagnosis. 

2. Description of the Prior Art 

Methods are already known which carry out an image 
registration on the basis of a distance criterion (Lisa 
Gottesfeld Brown, A Survey of Image Registration Techniques , 
ACM Computing Surveys, 24(4), 325-376, 1992; Jan Modersitzki, 
Numerical Methods for Image Registration, Habilitation , 
Institute of Mathematics, University of Luebeck, Germany, 
2002) . The general methodology is based on the optimization of 
a target function to be chosen in use-conforming manner and 
which is typically based on image intensities. In such 
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methods, apart from the image information, no further 
information is used for registration purposes. The 
registration result is only of an optimum nature in the sense 
of a global averaging. If particular significance is attached 
5 to specific, characteristic points in an application (such as 
e.g. the so-called anatomical landmarks in medical 
applications) , such methods cannot be recommended. 

Apart from image registration on the basis of a distance 
criterion, methods are also known which perform image 

10 registration exclusively on the basis of control points (Karl 
Rohr, Landmark-Based Image Analysis , Computational Imaging and 
Vision, Kluwer Academic Publishers, Dordrecht, 2001) . In such 
methods, prospectively or retrospectively, corresponding 
control points are associated with the views to be registered, 

15 and are then matched by means of registration. The 
disadvantage of such methods is that the registration 
exclusively takes account of control points. Such methods 
cannot take account of further image information, such as e.g. 
image intensities. In the case of unsatisfactory registration 

20 results, a user can only attempt to improve them by skilled 
introduction of further control points. The insertion of 
further control points is based on subjective trial and error 
for which no guidelines exist, and in particular, there is no 
automated procedure. 
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The problem of the invention is to develop an image 
registration method which leads both to a perfect, guaranteed 
correspondence between a number of predetermined control 
points, and also an optimum result in the sense of the distance 
5 criterion. 

SUMMARY OF THE INVENTION 
According to the invention, the problem is solved by the 
iterative determination of a transformation optimum with 
respect to a predetermined distance and smoothness criterion, 

10 in which control points corresponding in the images are imaged 
on one another in guaranteed manner by initializing an 
iteration counter and the initial displacement, determining the 
numerical solutions of the nonlinear partial differential 
equation (PDE) with the differential operator derivable from a 

15 predetermined smoothness criterion and the point evaluation 
functions located at the predetermined control points, 
combining the interpolation conditions, calculating a specific 
numeral solution of the PDE with the force determined on the 
basis of the distance criterion and the actual displacement 

20 field and the differential operator derived from the smoothness 
criterion, evaluating the specific solution at the control 
points, determining the coefficient for calculating an updated 
displacement, updating the displacement field and raising the 
iteration counter, checking the displacement for convergence, 
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and in the case of non-fulfillment of the convergence 
criterion, repetition of the previous steps. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Further advantages and features of the present invention 
5 can be gathered from the following description of the preferred 
embodiment relative to the attached drawing, wherein: 

FIG. 1 illustrates a schematic block diagram of the image 
registration method of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
10 |! The image registration method of the subject invention 

utilizes iterative determination of an optimum transformation 
with respect to a predetermined distance and smoothness 
criterion, wherein control points corresponding in the images 
can be imaged on one another in guaranteed manner, as shown in 
15 FIG. 1, by initializing the iteration counter and the initial 
displacement field 10, determining the numerical solutions of 
the nonlinear partial differential equation (PDE) for the 
differential operator and the point evaluation functions- 
located at the control points 20, combining the interpolation 
20 conditions 30, calculating a specific, numerical solution of 
the PDE for the differential operator and the force based on 
the actual displacement 40, evaluating the specific solution at 
the control points 50, determining the coefficients for 
calculating an updated displacement 60, updating the 
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displacement field and raising the iteration counter 70, and 
checking the displacement for convergence 80. In the event of 
non-fulfillment of the convergence criterion, the sequence is 
repeated. 

5 For simplification purposes, a view is referred to as a 

reference image (reference R) , and a further view, which is to 
be corrected, as a template (template T) . From the formal 
standpoint, these are functions of a d-dimensional, real space 
or a subset QcR d in the set of real numbers. Thus, to each d- 

10 dimensional point, xeQ is associated through R (x) and T(x) f a 
value which can be interpreted, e.g. as a color or grey value. 

In practical applications, particularly during every 
programming of the present method, the reference and template 
can be in discreet form. The images. are then functions on a 

15 lattice (e.g. Q={ 1, ni }x{ 1, n 2 ) for the dimension d=2) in a 
discreet set (e.g. in the set {0,...,255}) and can be 
interpreted as being formed. from pixels. For the registration 
method, these restrictions and, in particular, the specific 
nature of the discreetization are unimportant. The 

20 restrictions are solely made for simplified description 

purposes. The method can be used in the same way on random d- 
dimensional data sets. 

The function of image registration consists of the 
determination of a displacement function u, so that the 



requirement R (x) = T U (x) is optimum well fulfilled with the short 
form T u (x) :=T (x-u (x) ) for all xeQ. For calculating the 
template T u deformed by u in the case of discreet, 
predetermined images, such as are of a conventional nature in 
5 image processing, an interpolation (e.g. d-linear) has to be 
performed because the displaced coordinates x-u (x) are not 
necessarily located on the discreet lattice. The way in which 
such an interpolation takes place is unimportant for the 
registration method. 

10 Over and beyond the aforementioned similarity, 

requirements must be made on the displacement smoothness and on 
the imaging characteristics with respect to a number of 
preselected control points. In the simplest case, the 
coordinates of each of the m control points K Tfj of the template 

15 must be imaged on the, in each case, corresponding control 

point K*'^ of the reference, j=l,...,m. If the coordinates of the 
control points coincide, which may be ensurable by a 
preregistration, then u=0 applies at these points. 

As is normally the case with optimization problems, the 

20 determination of a minimizer of the aforementioned distance 
criterion can take place iteratively by means of a gradient 
descent method. In principle, any random distance criterion 
can be selected. The forces associate with the standard 
distance criteria are given in the literature (Modersitzki 



2002) • The specific nature of the calculation of these forces 
is unimportant for the registration method. 

In principle, any function known from the literature can 
be used as the smoothness criterion. From the smoothness 
5 criterion, it is possible to derive a partial differential 

operator A. These operators are known for the criteria used in 
the literature (Modersitzki 2002) . The sought displacement u 
can then be characterized as a solution of a nonlinear, partial 
differential equation (PDE) . 

10 For determining a numerical solution of this PDE, use is 

made of a finite differential approximation of the differential 
operator, which then leads to an equation system for the 
lattice values of the displacement. However, the specific 
discreetization of the differential equation lacks significance 

15 for the registration method. 

This procedure coincides with the method based solely on 
the distance criterion, and the smoothness criterion. The new 
aspect consists of a suitable binding in of the predetermined 
control points into the displacement calculation, in which a 

20 correspondence of the control points can be guaranteed. As 
methods are already known for the determination of the 
displacement based on the distance and smoothness criterion, a 
method is given here which combines partial solutions in an 
appropriate manner so as to give an overall solution, e.g. in 
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the form 

m 

u e (x)=v° (x)+ £ * e v j (x) , xeQ, e=l,...,d. 

e j-i j e 

A is the differential operator associated with the 
smoothness term and f is the field of forces belonging to the 
distance criterion, so that v° is a numerical solution of Av° = 
-f, the functions v j are numerical solutions of the 
distributional PDE Av j = 8 j , j = l,...,m, in which 5 j locates the 
point selection function (Dirac impulse) at control point K T,j . 
The specific, numerical method for the solution of the PDE is 
unimportant for the registration method. 

From the mathematical standpoint, v j , j=l,...,m are Green's 
functions of the differential, operator A, representing a 
solution of the PDE at the predetermined single point 
displacement. A suitable linear combination of these Green's 
functions consequently ensures that in the overall method, in 
the required manner, all the control points are imaged on one 
another . 

The function v° is so determined by means of an iterative 
method that the distance criterion is minimized, while 
maintaining the required smoothness. The weight functions A e j 
are so adapted that the control points are imaged in the 
required way. 
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The initialization of the program requires the selection 
of a distance and a smoothness criterion, or the force 
derivable from said criteria and the differential operator. On 
the basis of the point evaluation functions located at the 
5 control points, it is then possible to determine the Green's 
functions v j , j=l,...,m with a numerical method. They are not 
changed during the further course of the method. 

The inventive initialization is followed by a standard 
iteration procedure, during which there is a gradient descent, 

10 while taking account of the control points. No human 

intervention is needed. > Thus, the described method combines 
the advantages of methods based on distance criteria 
(particularly, automatability and on an average optimum 
registration) with those of the control point method 

15 (guaranteed registration of distinguished points) , and on 
predetermining an initial set of control points, give 
reproducible, optimum results independent of the user or 
computer program. No important part is played by computer code 
details in the final result of the image registration, and they 

20 only influence the requisite computing time and memory 
requirements . 

The images to be registered can be digital images, pixels, 
JPEG, wavelet-based objects or acoustic signals, 
i The linear equation systems occurring in the method can be 
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solved directly, indirectly, iteratively or by multigrid, and 
for the method use can be made of a reference coordinate system 
imaged by Euler or Lagrange coordinates. 

The invention also proposes the registration of one, two 
5 or three-dimensional, and sequences of one, two or three- 
dimensional objects, as well as the use of control points in 
the form of anatomical landmarks, fiduciary markers or other 
characteristic quantities . 

The distance criterion proposed is based on intensity, 
10 edge, corner, surface normal or level set or on the "sum of 
squared differences", L 2 distance, correlation, correlation 
variants, mutual information or mutual information variants. 

The force terms associated with the distance quantity are 
to be calculated by finite difference methods or gradient 
15 formation, and the smoothness criterion used to be physically 
motivated by means of an elastic potential, or a fluid approach 
or diffusive or curvature approaches based on time or space 
derivatives of the displacement. 

The boundary conditions of the differential operator are 
20 advantageously given by explicit or implicit, Neumann, 

Dirichlet, sliding, bending or periodic boundary conditions. 

The nature of the discreetization of the differential 
operator should be based on finite differences, finite volume, 
finite elements, Fourier methods, series expansions, filter 
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techniques, collocations or multigrid, and interpolation is to 
be performed d-dimensionally by means of splines or wavelets. 

Finally, displacement can be explicitly updated by means 
of the increment of the displacement or its time derivative. 
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AMENDED CLAIMS 

Having thus described the invention, what is claimed and 
desired to be secured by Letters Patent is: 
Claims 1-10 (canceled) . 

11. (new) A method for registration of images by iterative 
determination of an optimum transformation with respect to a 
predetermined distance and smoothness criterion, wherein 
control points corresponding in the images can be imaged on one 
another in guaranteed manner, the steps of the image 
registration method comprising: 

(a) initializing an iteration counter and 
displacement field; 

(b) determining the numeral solutions of the 
nonlinear partial differential equation (PDE) with the 
differential operator derivable from a predetermined 
smoothness criterion and the point evaluation functions 
located at the predetermined control points; 

(c) combining the interpolation conditions; 

(d) calculating a specific numerical solution of the 
partial differential equation (PDE) with the force 
determined on the basis of the distance criterion and the 
actual displacement field, the differential operator 
derived from the smoothness criterion; 
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(e) evaluating the specific solution at the control 
points; 

(f) determining the coefficients for calculating an 
updated displacement; 

(g) updating the displacement field and raising the 
iteration counter; and 

(h) checking the displacement for convergence. 

12. (new) The image registration method of Claim 11, wherein 
steps (d) through (h) are repeated for non-fulfillment of the 
convergence criterion . 

13. (new) The image registration method of Claim 11, wherein 
the interpolation is performed d-dimensionally by means of 
splines or wavelets. 

14. (new) The image registration, method of Claim 11, wherein 
displacement is updated by means of the increment of the 
displacement or its time derivative. 

15. (new) The image registration method of Claim 11, wherein 
one, two or three-dimensional objects are registered. 

16. (new) The image registration method of Claim 15, wherein 
the sequences of one, two or three-dimensional objects are 
registered . 

17. (new) The image registration method of Claim 11, wherein 
control points are characterized by anatomical landmarks, 
fiduciary markers or other quantities. 
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18. (new) The image registration method of Claim 11, wherein 
the distance criterion is based on intensity, edge, corner, 
surface normal or level set, sum of square differences, L 2 
distance, correlation, mutual information or other variants. 

19. (new) The image registration method of Claim 11, wherein 
force terms of distance quantity are calculated by means of 
finite difference methods or gradient formation. 

20. (new) The image registration method of Claim 11, wherein 
smoothness criterion is enforced by means of elastic potential, 
fluid approach, or diffusive or curvature approaches based on 
time and space derivatives. 

21. (new) The image registration method of Claim 11, wherein 
boundary conditions of the differential operator are specified 
as explicit or implicit, Neumann, Dirichlet, sliding, bending 
or periodic. 

22. (new) The image registration method of Claim 19, wherein 
discreetization of the differential operator is based on finite 
differences, finite volume, finite elements, Fourier methods, 
series expansions, filter techniques, collocations or 
multigrid. 
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IMAGE REGISTRATION METHOD 
ABSTRACT 

An image registration method iteratively determining a 
transformation that is optimal regarding a given distance 
criterion and smoothness criterion. The method allowing 
corresponding landmarks in the images to be definitely 
represented on top of each other, the registered images being 
derivative of digital images, pixels, JPEG, wavelet-based or 
acoustic signals. 
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INITIALIZATION OF THE ITERATION COUNTER AND 
THE INITIAL DISPLACEMENT FIELD 
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NUMERICAL SOLUTION OF THE PDE FOR THE DIFFERENTIAL OPERATOR AND 
THE POINT EVALUATION FUNCTIONS LOCATED AT THE CONTROL POINTS 
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COMBINING THE INTERPOLATION 
CONDITIONS 



CALCULATION OF A SPECIFIC, NUMERICAL SOLUTION OF THE 
PDE FOR THE DIFFERENTIAL OPERATOR AND THE FORCE 
BASED ON THE ACTUAL DISPLACEMENT 
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EVALUATION OF THE SPECIFIC SOLUTION 
AT THE CONTROL POINTS 
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DETERMINATION OF THE COEFFICIENTS FOR 
CALCULATING AN UPDATED DISPLACEMENT 
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UPDATING THE DISPLACEMENT FIELD 
AND RAISING THE ITERATION COUNTER 
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CHECKING THE DISPLACEMENT FOR 
CONVERGENCE 
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